Variable Selection is Hard

Variable selection for sparse linear regression is the problem of finding, given anm×pmatrixB and a target vector y, a sparse vector x such thatBx approximately equals y. Assuming a standard complexity hypothesis, we show that no polynomial-time algorithm can find a k′-sparse x with ||Bx − y|| ≤ h(m, p), where k′ = k ·2log p and h(m, p) ≤ pC1m1−C2 , where δ > 0, C1 > 0, C2 > 0 are arbitrary. This is true even under the promise that there is an unknown k-sparse vector x∗ satisfying Bx∗ = y. We prove a similar result for a statistical version of the problem in which the data are corrupted by noise. To the authors’ knowledge, these are the first hardness results for sparse regression that apply when the algorithm simultaneously has k′ > k and h(m, p) > 0.